<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Yocto Project Test Environment Manual</title><link rel="stylesheet" type="text/css" href="test-manual-style.css" /><meta name="generator" content="DocBook XSL Stylesheets V1.76.1" /></head><body><div xml:lang="en" class="book" title="Yocto Project Test Environment Manual" id="test-manual" lang="en"><div class="titlepage"><div><div><h1 class="title">
            Yocto Project Test Environment Manual
        </h1></div><div><div class="authorgroup">
            <div class="author"><h3 class="author"><span class="firstname">Scott</span> <span class="surname">Rifenbark</span></h3><div class="affiliation">
                    <span class="orgname">Scotty's Documentation Services, INC<br /></span>
                </div><code class="email">&lt;<a class="email" href="mailto:srifenbark@gmail.com">srifenbark@gmail.com</a>&gt;</code></div>
        </div></div><div><p class="copyright">Copyright © 2010-2018 Linux Foundation</p></div><div><div class="legalnotice" title="Legal Notice"><a id="idm45709743826400"></a>
      <p>
          Permission is granted to copy, distribute and/or modify this document under
          the terms of the <a class="ulink" href="http://creativecommons.org/licenses/by-sa/2.0/uk/" target="_top">
          Creative Commons Attribution-Share Alike 2.0 UK: England &amp; Wales</a> as published by
          Creative Commons.
      </p>
           <div class="note" title="Manual Notes" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Manual Notes</h3>
               <div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
                       This version of the
                       <span class="emphasis"><em>Yocto Project Test Environment Manual</em></span>
                       is for the 2.6 release of the
                       Yocto Project.
                       To be sure you have the latest version of the manual
                       for this release, go to the
                       <a class="ulink" href="http://www.yoctoproject.org/documentation" target="_top">Yocto Project documentation page</a>
                       and select the manual from that site.
                       Manuals from the site are more up-to-date than manuals
                       derived from the Yocto Project released TAR files.
                       </p></li><li class="listitem"><p>
                       If you located this manual through a web search, the
                       version of the manual might not be the one you want
                       (e.g. the search might have returned a manual much
                       older than the Yocto Project version with which you
                       are working).
                       You can see all Yocto Project major releases by
                       visiting the
                       <a class="ulink" href="https://wiki.yoctoproject.org/wiki/Releases" target="_top">Releases</a>
                       page.
                       If you need a version of this manual for a different
                       Yocto Project release, visit the
                       <a class="ulink" href="http://www.yoctoproject.org/documentation" target="_top">Yocto Project documentation page</a>
                       and select the manual set by using the
                       "ACTIVE RELEASES DOCUMENTATION" or "DOCUMENTS ARCHIVE"
                       pull-down menus.
                       </p></li><li class="listitem"><p>
                       To report any inaccuracies or problems with this
                       manual, send an email to the Yocto Project
                       discussion group at
                       <code class="filename">yocto@yoctoproject.com</code> or log into
                       the freenode <code class="filename">#yocto</code> channel.
                       </p></li></ul></div>
           </div>
    </div></div><div><div class="revhistory"><table border="1" width="100%" summary="Revision history"><tr><th align="left" valign="top" colspan="2"><strong>Revision History</strong></th></tr>
            <tr><td align="left">Revision 2.7</td><td align="left">TBD</td></tr><tr><td align="left" colspan="2">Released with the Yocto Project 2.7 Release.</td></tr>
        </table></div></div></div><hr /></div><div class="toc"><p><strong>Table of Contents</strong></p><dl><dt><span class="chapter"><a href="#test-manual-intro">1. The Yocto Project Test Environment Manual</a></span></dt><dd><dl><dt><span class="section"><a href="#test-welcome">1.1. Welcome</a></span></dt><dt><span class="section"><a href="#test-yocto-project-autobuilder-overview">1.2. Yocto Project Autobuilder Overview</a></span></dt><dt><span class="section"><a href="#test-project-tests">1.3. Yocto Project Tests - Type Overview</a></span></dt><dt><span class="section"><a href="#test-test-mapping">1.4. How Tests Map to Areas of Code</a></span></dt><dt><span class="section"><a href="#test-examples">1.5. Test Examples</a></span></dt><dd><dl><dt><span class="section"><a href="#bitbake-selftest-example">1.5.1. <code class="filename">bitbake-selftest</code></a></span></dt><dt><span class="section"><a href="#oe-selftest-example">1.5.2. <code class="filename">oe-selftest</code></a></span></dt><dt><span class="section"><a href="#testimage-example">1.5.3. <code class="filename">testimage</code></a></span></dt><dt><span class="section"><a href="#testsdk_ext-example">1.5.4. <code class="filename">testsdk_ext</code></a></span></dt><dt><span class="section"><a href="#testsdk-example">1.5.5. <code class="filename">testsdk</code></a></span></dt><dt><span class="section"><a href="#oe-build-perf-test-example">1.5.6. <code class="filename">oe-build-perf-test</code></a></span></dt></dl></dd><dt><span class="section"><a href="#some-id">1.6. New Section on the Periodic Builds</a></span></dt><dt><span class="section"><a href="#test-configuring-and-triggering-autobuilder-helper-build-scripts">1.7. Configuring and Triggering Autobuilder Helper Build Scripts</a></span></dt><dt><span class="section"><a href="#test-deploying-yocto-autobuilder">1.8. Deploying Yocto Autobuilder</a></span></dt><dd><dl><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-controller">1.8.1. Upstream Autobuilder Deployment on the Controller</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-worker">1.8.2. Upstream Autobuilder Deployment on the Worker</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-no-upstream-users">1.8.3. Upstream Autobuilder Deployment No Upstream Users</a></span></dt></dl></dd><dt><span class="section"><a href="#test-headless-sanity-tests">1.9. Setting Up Headless Sanity Tests</a></span></dt><dt><span class="section"><a href="#test-adding-additional-build-workers">1.10. Adding Additional Build Workers</a></span></dt><dt><span class="section"><a href="#test-setting-up-build-history">1.11. Setting Up Build History</a></span></dt><dt><span class="section"><a href="#test-some-more-notes">1.12. Some More Notes</a></span></dt><dt><span class="section"><a href="#test-yocto-project-helper">1.13. Yocto Project Autobuilder Helper Scripts</a></span></dt></dl></dd></dl></div>
    

    <div class="chapter" title="Chapter 1. The Yocto Project Test Environment Manual" id="test-manual-intro"><div class="titlepage"><div><div><h2 class="title">Chapter 1. The Yocto Project Test Environment Manual<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-manual-intro">¶</a></span></h2></div></div></div><div class="toc"><p><strong>Table of Contents</strong></p><dl><dt><span class="section"><a href="#test-welcome">1.1. Welcome</a></span></dt><dt><span class="section"><a href="#test-yocto-project-autobuilder-overview">1.2. Yocto Project Autobuilder Overview</a></span></dt><dt><span class="section"><a href="#test-project-tests">1.3. Yocto Project Tests - Type Overview</a></span></dt><dt><span class="section"><a href="#test-test-mapping">1.4. How Tests Map to Areas of Code</a></span></dt><dt><span class="section"><a href="#test-examples">1.5. Test Examples</a></span></dt><dd><dl><dt><span class="section"><a href="#bitbake-selftest-example">1.5.1. <code class="filename">bitbake-selftest</code></a></span></dt><dt><span class="section"><a href="#oe-selftest-example">1.5.2. <code class="filename">oe-selftest</code></a></span></dt><dt><span class="section"><a href="#testimage-example">1.5.3. <code class="filename">testimage</code></a></span></dt><dt><span class="section"><a href="#testsdk_ext-example">1.5.4. <code class="filename">testsdk_ext</code></a></span></dt><dt><span class="section"><a href="#testsdk-example">1.5.5. <code class="filename">testsdk</code></a></span></dt><dt><span class="section"><a href="#oe-build-perf-test-example">1.5.6. <code class="filename">oe-build-perf-test</code></a></span></dt></dl></dd><dt><span class="section"><a href="#some-id">1.6. New Section on the Periodic Builds</a></span></dt><dt><span class="section"><a href="#test-configuring-and-triggering-autobuilder-helper-build-scripts">1.7. Configuring and Triggering Autobuilder Helper Build Scripts</a></span></dt><dt><span class="section"><a href="#test-deploying-yocto-autobuilder">1.8. Deploying Yocto Autobuilder</a></span></dt><dd><dl><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-controller">1.8.1. Upstream Autobuilder Deployment on the Controller</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-worker">1.8.2. Upstream Autobuilder Deployment on the Worker</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-no-upstream-users">1.8.3. Upstream Autobuilder Deployment No Upstream Users</a></span></dt></dl></dd><dt><span class="section"><a href="#test-headless-sanity-tests">1.9. Setting Up Headless Sanity Tests</a></span></dt><dt><span class="section"><a href="#test-adding-additional-build-workers">1.10. Adding Additional Build Workers</a></span></dt><dt><span class="section"><a href="#test-setting-up-build-history">1.11. Setting Up Build History</a></span></dt><dt><span class="section"><a href="#test-some-more-notes">1.12. Some More Notes</a></span></dt><dt><span class="section"><a href="#test-yocto-project-helper">1.13. Yocto Project Autobuilder Helper Scripts</a></span></dt></dl></div><div class="section" title="1.1. Welcome"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-welcome">1.1. Welcome<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-welcome">¶</a></span></h2></div></div></div><p>
            Welcome to the Yocto Project Test Environment Manual!
            This manual is a work in progress.
            The manual contains information about the testing environment
            used by the Yocto Project to make sure each major and minor
            release works as planned.
            Other organizations can leverage off the process and testing
            environment used by the Yocto Project to create their own
            automated, production test environment.
        </p><p>
            Currently, the Yocto Project Test Environment Manual has no
            projected release date.
            This manual is a work-in-progress and is being initially loaded
            with information from the README files and notes from key
            engineers:
            </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
                    <span class="emphasis"><em><code class="filename">yocto-autobuilder</code>:</em></span>
                    This
                    <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/README" target="_top"><code class="filename">README.md</code></a>
                    is not maintained.
                    However, some information from this README file still
                    applies but could need some modification.
                    In particular, information about setting up headless
                    sanity tests and build history.
                    The sections on these will be changing.
                    </p><div class="note" title="IMPORTANT" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">IMPORTANT</h3>
                        The <code class="filename">yocto-autobuilder</code> 
                        <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/" target="_top">repository</a>
                        is obsolete and is no longer maintained.
                        The new "Autobuilder" is maintained in the
                        <code class="filename">yocto-autobuilder2</code> 
                        <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/" target="_top">repository</a>.
                    </div><p>
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em><code class="filename">yocto-autobuilder2</code>:</em></span>
                    This
                    <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/README.md" target="_top"><code class="filename">README.md</code></a>
                    is the main README for Yocto Project Autobuilder.
                    The <code class="filename">yocto-autobuilder2</code> repository
                    represents the Yocto Project's testing codebase and
                    exists to configure and use Buildbot to do testing.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em><code class="filename">yocto-autobuilder-helper</code>:</em></span>
                    This
                    <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree/README" target="_top"><code class="filename">README</code></a>
                    is a valid Autobuilder Git repository that contains
                    Yocto Project Autobuilder Helper Scripts.
                    The <code class="filename">yocto-autobuilder-helper</code>
                    repository contains the "glue" logic that connects any
                    Continuous Improvement (CI) system to run builds,
                    support getting the correct code revisions, configure
                    builds and layers, run builds, and collect results.
                    The code is independent of any CI system, which means
                    the code can work Buildbot, Jenkins, or others.
                    </p></li></ul></div><p>
        </p></div><div class="section" title="1.2. Yocto Project Autobuilder Overview"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-yocto-project-autobuilder-overview">1.2. Yocto Project Autobuilder Overview<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-yocto-project-autobuilder-overview">¶</a></span></h2></div></div></div><p>
            The Yocto Project Autobuilder collectively refers to the software,
            tools, scripts, and procedures used by the Yocto Project to test
            released software across supported hardware in an automated and
            regular fashion.
            Basically, during the development of a Yocto Project release, the
            Autobuilder tests if things work.
            The Autobuilder builds all test targets and runs all the tests.
        </p><p>
            The Yocto Project uses the unpatched
            <a class="ulink" href="https://docs.buildbot.net/0.9.15.post1/" target="_top">Buildbot Nine</a>
            to drive its integration and testing.
            Buildbot Nine has a plug-in interface that the Yocto Project
            customizes using code from the
            <code class="filename">yocto-autobuilder2</code> repository.
            The resulting customized UI plug-in allows you to
            visualize builds in a way suited to the project.
        </p><p>
            A "helper" layer provides configuration and job management
            through scripts found in the
            <code class="filename">yocto-autobuilder-helper</code> repository.
            The helper layer contains the bulk of the build configuration
            information and is release-specific, which makes it highly
            customizable on a per-project basis.
            The layer is CI system-agnostic and contains a number of helper
            scripts that can generate build configurations from simple JSON
            files.
            </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
                It is possible to use the outer layers from another
                Continuous Integration (CI) system such as
                <a class="ulink" href="https://en.wikipedia.org/wiki/Jenkins_(software)" target="_top">Jenkins</a>
                instead of Buildbot.
            </div><p>
        </p><p>
            The following figure shows the Yocto Project Autobuilder stack
            with a topology that includes a controller and a cluster of
            workers:
            </p><table border="0" summary="manufactured viewport for HTML img" cellspacing="0" cellpadding="0" width="720"><tr style="height: 540px"><td align="center"><img src="figures/ab-test-stack.png" align="middle" width="720" /></td></tr></table><p>
        </p></div><div class="section" title="1.3. Yocto Project Tests - Type Overview"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-project-tests">1.3. Yocto Project Tests - Type Overview<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-project-tests">¶</a></span></h2></div></div></div><p>
            Two kinds of tests exist within the Yocto Project:
            </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
                    <span class="emphasis"><em>Build Testing:</em></span>
                    Tests whether specific configurations build by varying
                    <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-MACHINE" target="_top"><code class="filename">MACHINE</code></a>,
                    <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-DISTRO" target="_top"><code class="filename">DISTRO</code></a>,
                    and the specific target images being built (or world).
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>Build Performance Testing:</em></span>
                    Tests whether or not commonly used steps during builds work
                    efficiently and avoid regressions.
                    </p></li></ul></div><p>
            Beyond these types of testing, the Autobuilder tests different
            pieces by using the following types of tests:
            </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
                    <span class="emphasis"><em>Build Testing:</em></span>
                    Trigger builds of all the different test configurations
                    on the Autobuilder.
                    Builds usually cover each target for different
                    architectures, machines, and distributions.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>Build Performance Testing:</em></span>
                    Tests to time commonly used usage scenarios are run through
                    <code class="filename">oe-build-perf-test</code>.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>eSDK Testing:</em></span>
                    Image tests initiated through the following command:
                    </p><pre class="literallayout">
     $ bitbake <em class="replaceable"><code>image</code></em> -c testsdkext
                    </pre><p>
                    The tests utilize the <code class="filename">testsdkext</code>
                    class and the <code class="filename">do_testsdkext</code> task.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>Feature Testing:</em></span>
                    Various scenario-based tests are run through the
                    <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#testing-and-quality-assurance" target="_top">OpenEmbedded Self-Test</a>
                    (oe-selftest).
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>Image Testing:</em></span>
                    Image tests initiated through the following command:
                    </p><pre class="literallayout">
     $ bitbake <em class="replaceable"><code>image</code></em> -c testimage
                    </pre><p>
                    The tests utilize the
                    <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-classes-testimage*" target="_top"><code class="filename">testimage*</code></a>
                    classes and the
                    <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-tasks-testimage" target="_top"><code class="filename">do_testimage</code></a>
                    task.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>Package Testing:</em></span>
                    A Package Test (ptest) runs tests against packages built
                    by the OpenEmbedded build system on the target machine.
                    See the
                    "<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/dev-manual/dev-manual.html#testing-packages-with-ptest" target="_top">Testing Packages With ptest</a>"
                    section in the Yocto Project Development Tasks Manual and
                    the
                    "<a class="ulink" href="https://wiki.yoctoproject.org/wiki/Ptest" target="_top">Ptest</a>"
                    Wiki page for more information on Ptest.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>Sanity Checks During the Build Process:</em></span>
                    Tests initiated through the
                    <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-classes-insane" target="_top"><code class="filename">insane</code></a>
                    class.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>SDK Testing:</em></span>
                    Image tests initiated through the following command:
                    </p><pre class="literallayout">
     $ bitbake <em class="replaceable"><code>image</code></em> -c testsdk
                    </pre><p>
                    The tests utilize the
                    <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-classes-testsdk" target="_top"><code class="filename">testsdk</code></a>
                    class and the <code class="filename">do_testsdk</code> task.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>Unit Testing:</em></span>
                    Unit tests on various components of the system run
                    through <code class="filename">oe-selftest</code> and
                    <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#testing-and-quality-assurance" target="_top"><code class="filename">bitbake-selftest</code></a>.
                    </p></li></ul></div><p>
        </p></div><div class="section" title="1.4. How Tests Map to Areas of Code"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-test-mapping">1.4. How Tests Map to Areas of Code<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-test-mapping">¶</a></span></h2></div></div></div><p>
            Tests map into the codebase as follows:
            </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
                    <span class="emphasis"><em>bitbake-selftest:</em></span>
                    </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
                            These tests are self-contained and test BitBake
                            as well as its APIs, which include the fetchers.
                            The tests are located in
                            <code class="filename">bitbake/lib/*/tests</code>.
                            </p></li><li class="listitem"><p>
                            From within the BitBake repository, run the
                            following:
                            </p><pre class="literallayout">
     $ bitbake-selftest
                            </pre><p>
                            </p></li><li class="listitem"><p>
                            The tests are based on
                            <a class="ulink" href="https://docs.python.org/3/library/unittest.html" target="_top">Python unittest</a>.
                            </p></li></ul></div><p>
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>oe-selftest:</em></span>
                    </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
                            These tests use OE to test the workflows, which
                            include testing specific features, behaviors
                            of tasks, and API unit tests.
                            The tests take advantage of parallelism through
                            the "-j" option to run in multiple threads.
                            </p></li><li class="listitem"><p>
                            The tests are based on Python unittest.
                            </p></li><li class="listitem"><p>
                            The code for the tests resides in
                            <code class="filename">meta/lib/oeqa/selftest</code>.
                            </p></li><li class="listitem"><p>
                            To run all the test, enter the following command:
                            </p><pre class="literallayout">
     $ oe-selftest -a
                            </pre><p>
                            </p></li><li class="listitem"><p>
                            To run a specific test, use the following command
                            form where <em class="replaceable"><code>testname</code></em> is
                            the name of the specific test:
                            </p><pre class="literallayout">
     $ oe-selftest -r <em class="replaceable"><code>testname</code></em>
                            </pre><p>
                            </p></li></ul></div><p>
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>testimage:</em></span>
                    </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
                            These tests build an image, boot it, and run tests
                            against the image's content.
                            </p></li><li class="listitem"><p>
                            The code for these tests resides in
                            <code class="filename">meta/lib/oeqa/runtime</code>.
                            </p></li><li class="listitem"><p>
                            You need to set the
                            <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-IMAGE_CLASSES" target="_top"><code class="filename">IMAGE_CLASSES</code></a>
                            variable as follows:
                            </p><pre class="literallayout">
     IMAGE_CLASSES += "testimage"
                            </pre><p>
                            </p></li><li class="listitem"><p>
                            Run the tests using the following command form:
                            </p><pre class="literallayout">
     $ bitbake <em class="replaceable"><code>image</code></em> -c testimage
                            </pre><p>
                            </p></li></ul></div><p>
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>testsdk:</em></span>
                    </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
                            These tests build an SDK, install it, and then
                            run tests against that SDK.
                            </p></li><li class="listitem"><p>
                            The code for these tests resides in
                            <code class="filename">meta/lib/oeqa/sdk</code>.
                            </p></li><li class="listitem"><p>
                            Run the test using the following command form:
                            </p><pre class="literallayout">
     $ bitbake <em class="replaceable"><code>image</code></em> -c testsdk
                            </pre><p>
                            </p></li></ul></div><p>
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>testsdk_ext:</em></span>
                    </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
                            These tests build an extended SDK (eSDK), install
                            that eSDK, and run tests against the eSDK.
                            </p></li><li class="listitem"><p>
                            The code for these tests resides in
                            <code class="filename">meta/lib/oeqa/esdk</code>.
                            </p></li><li class="listitem"><p>
                            To run the tests, use the following command form:
                            </p><pre class="literallayout">
     $ bitbake <em class="replaceable"><code>image</code></em> -c testsdkext
                            </pre><p>
                            </p></li></ul></div><p>
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>oe-build-perf-test:</em></span>
                    </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
                            These tests run through commonly used usage
                            scenarios and measure the performance times.
                            </p></li><li class="listitem"><p>
                            The code for these tests resides in
                            NEED A DIRECTORY HERE.
                            </p></li><li class="listitem"><p>
                            NEED SOME INFORMATION ON HOW TO ENABLE THIS
                            TEST OR INCLUDE IT HERE.
                            </p><pre class="literallayout">
     <em class="replaceable"><code>some setting</code></em>
                            </pre><p>
                            </p></li><li class="listitem"><p>
                            Run the tests using the following command form:
                            </p><pre class="literallayout">
     $ <em class="replaceable"><code>some command</code></em>
                            </pre><p>
                            </p></li></ul></div><p>
                    </p></li></ul></div><p>
        </p></div><div class="section" title="1.5. Test Examples"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-examples">1.5. Test Examples<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-examples">¶</a></span></h2></div></div></div><p>
            This section provides example tests for each of the tests
            listed in the
            <a class="link" href="#test-test-mapping" title="1.4. How Tests Map to Areas of Code">How Tests Map to Areas of Code</a>"
            section.
        </p><div class="section" title="1.5.1. bitbake-selftest"><div class="titlepage"><div><div><h3 class="title" id="bitbake-selftest-example">1.5.1. <code class="filename">bitbake-selftest</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#bitbake-selftest-example">¶</a></span></h3></div></div></div><p>
                Content here.
            </p></div><div class="section" title="1.5.2. oe-selftest"><div class="titlepage"><div><div><h3 class="title" id="oe-selftest-example">1.5.2. <code class="filename">oe-selftest</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#oe-selftest-example">¶</a></span></h3></div></div></div><p>
                NEED CONTENT HERE.
            </p></div><div class="section" title="1.5.3. testimage"><div class="titlepage"><div><div><h3 class="title" id="testimage-example">1.5.3. <code class="filename">testimage</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#testimage-example">¶</a></span></h3></div></div></div><p>
                NEED CONTENT HERE.
            </p></div><div class="section" title="1.5.4. testsdk_ext"><div class="titlepage"><div><div><h3 class="title" id="testsdk_ext-example">1.5.4. <code class="filename">testsdk_ext</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#testsdk_ext-example">¶</a></span></h3></div></div></div><p>
                NEED CONTENT HERE.
            </p></div><div class="section" title="1.5.5. testsdk"><div class="titlepage"><div><div><h3 class="title" id="testsdk-example">1.5.5. <code class="filename">testsdk</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#testsdk-example">¶</a></span></h3></div></div></div><p>
                NEED CONTENT HERE.
            </p></div><div class="section" title="1.5.6. oe-build-perf-test"><div class="titlepage"><div><div><h3 class="title" id="oe-build-perf-test-example">1.5.6. <code class="filename">oe-build-perf-test</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#oe-build-perf-test-example">¶</a></span></h3></div></div></div><p>
                NEED CONTENT HERE.
            </p></div></div><div class="section" title="1.6. New Section on the Periodic Builds"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="some-id">1.6. New Section on the Periodic Builds<span class="permalink"><a alt="Permalink" title="Permalink" href="#some-id">¶</a></span></h2></div></div></div><p>
            The following is going to be the replacement content for the
            section on "Nightly Builds".
            Not sure what we are going to call these builds.
            We need a name to replace "Nightly Builds".
        </p><p>
            Here is the content from Richards email:
            </p><pre class="literallayout">
     In 1.6, we actually dropped the "nightly" bit pretty much everywhere.
     They are now named MACHINE or MACHINE-DISTRO, e.g. qemuarm or qemuarm-
     lsb (which tests poky-lsb with qemuarm). We now parallelise not just
     architecture but by machine so machine and real hardware are now
     separate. The flow is therefore to build the images+sdks, then test the
     images+sdks, trying to do as much as possible in parallel.

     We have two types of build trigger, "quick" and "full". quick runs all
     the things which commonly fail and one random oe-selftest. "full" runs
     all our targets, runs oe-selftest on all distros and includes ptest and
     build performance tests. Its slower but more complete and would be used
     for release builds.
            </pre><p>
        </p></div><div class="section" title="1.7. Configuring and Triggering Autobuilder Helper Build Scripts"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-configuring-and-triggering-autobuilder-helper-build-scripts">1.7. Configuring and Triggering Autobuilder Helper Build Scripts<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-configuring-and-triggering-autobuilder-helper-build-scripts">¶</a></span></h2></div></div></div><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
            This section is created from the information in the
            <code class="filename">yocto-autobuilder2</code> 
            <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/README.md" target="_top"><code class="filename">README.md</code></a>
            file.
            I am making an assumption that we do not want to refer to the
            Autobuilder stuff as "Autobuilder2".
            My guess is that since this is the first documentation of any
            automated test environment and process in the Yocto Project
            user documentation, we will treat it as the start of things.
        </div><p>
            Automatic testing is based on the workers executing builds using
            Buildbot Nine configured for specific build jobs triggered in an
            automatic and regular fashion.
            Worker Configuration and triggering is accomplished through
            the
            <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/" target="_top">Yocto Project Autobuilder layer</a>
            and a set of
            <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree" target="_top">helper scripts</a>.
        </p><p>
            The configuration and helper scripts have as little code and
            as few custom Buildbot extensions as possible.
            The configuration collects required input from the user to
            furnish the helper scripts with the input needed for workers
            to accomplish their builds.
            The input consists of minimal user-customizable parameters
            used to trigger the helper build scripts.
        </p><p>
            Each builder maps to a named configuration in the helper
            scripts.
            The configuration is created with the steps and properties
            required to invoke the helper scripts for a worker's builds.
        </p><p>
            Each worker has a custom scheduler created for it and contains
            parameters configured for the scheduler that can supply the custom
            versions of the required values for the helper script parameters.
        </p><p>
            Following is the code layout for the Autobuilder:
            </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
                    <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/builders.py" target="_top"><code class="filename">builders.py</code></a>:</em></span>
                    Configures the builders with the minimal buildsteps
                    to invoke the Yocto Project Autobuilder helper scripts.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/lib/wiki.py" target="_top"><code class="filename">lib/wiki.py</code></a>:</em></span>
                    Implements functionality related to
                    <a class="ulink" href="https://www.mediawiki.org/wiki/MediaWiki" target="_top">MediaWiki</a>.
                    The <code class="filename">wikilog</code> plug-in uses this
                    functionality.
                    Effectively, this functionality provides helper functions
                    for the plug-in.
                    </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
                        Much of this code can be replaced by porting the
                        plug-in so that it is implemented as a
                        <code class="filename">buildbot.util.service.HTTPClient</code>.
                    </div><p>
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/reporters/wikilog.py" target="_top"><code class="filename">reporters/wikilog.py</code></a>:</em></span>
                    A custom plug-in that is a Buildbot service that listens for
                    build failures and then writes information about the
                    failure to the configured wiki page.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/steps/writelayerinfo.py" target="_top"><code class="filename">steps/writelayerinfo.py</code></a>:</em></span>
                    Implements a simple, custom buildset that iterates the
                    <code class="filename">repo_</code>, <code class="filename">branch_</code>,
                    and <code class="filename">commit_</code> properties, which are set
                    by the schedulers, and then writes a JSON file with the
                    user's values.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/config.py" target="_top"><code class="filename">config.py</code></a>:</em></span>
                    Contains all values that might need changing to redeploy
                    the Autobuilder code elsewhere.
                    </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
                        The redeployment goal has not been currently met.
                    </div><p>
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/master.cfg" target="_top"><code class="filename">master.cfg</code></a>:</em></span>
                    Performs most configuration by making calls into other
                    scripts.
                    Configuration specific for a worker cluster (i.e. a
                    Controller URL) resides here.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/schedulers.py" target="_top"><code class="filename">schedulers.py</code></a>:</em></span>
                    Sets up the force schedulers with controls for modifying
                    inputs for each worker.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/services.py" target="_top"><code class="filename">services.py</code></a>:</em></span>
                    Configures IRC, mail, and Wikilog reporters.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/workers.py" target="_top"><code class="filename">workers.py</code></a>:</em></span>
                    Configures the worker objects.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/www.py" target="_top"><code class="filename">www.py</code></a>:</em></span>
                    Sets up the Web User Interface.
                    </p></li></ul></div><p>
        </p><p>
            The goal is to keep custom code minimized throughout the
            Autobuilder.
            The few customizations implemented support the Yocto Project
            Autobuilder Helper Script workflows and help replicate the
            workflows established with the Yocto Autobuilder layer.
            In particular, the following files accomplish this customization:
            </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
                    <code class="filename">writelayerinfo.py</code>
                    </p></li><li class="listitem"><p>
                    <code class="filename">wikilog.py</code>
                    </p></li><li class="listitem"><p>
                    <code class="filename">wiki.py</code>
                    </p></li></ul></div><p>
        </p></div><div class="section" title="1.8. Deploying Yocto Autobuilder"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-deploying-yocto-autobuilder">1.8. Deploying Yocto Autobuilder<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-deploying-yocto-autobuilder">¶</a></span></h2></div></div></div><p>
            Steps to deploy the Yocto Project Autobuilder assume each target
            system has a copy of Buildbot installed.
            Additionally, various pieces of functionality require that a copy
            of the Autobuilder Helper Scripts (i.e.
            <code class="filename">yocto-autobuilder-helper</code>) is available
            in the home directory at
            <code class="filename">~/yocto-autobuilder-helper</code> of the user
            running Buildbot.
            </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
                If you are using a reverse proxy, be aware that modern
                Buildbot uses a  web socket for various communications between
                the master and the web's User Interface.
                Refer to the
                <a class="ulink" href="http://docs.buildbot.net/latest/manual/cfg-www.html#reverse-proxy-configuration" target="_top">Buildbot documentation</a>
                for information on how to correctly configure a reverse proxy.
            </div><p>
        </p><p>
            The following sections provide steps for Yocto Autobuilder
            deployment.
        </p><div class="section" title="1.8.1. Upstream Autobuilder Deployment on the Controller"><div class="titlepage"><div><div><h3 class="title" id="test-upstream-autobuilder-deployment-on-the-controller">1.8.1. Upstream Autobuilder Deployment on the Controller<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-upstream-autobuilder-deployment-on-the-controller">¶</a></span></h3></div></div></div><p>
                Follow these steps to deploy Yocto Autobuilder on an
                upstream controller:
                </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
                        <span class="emphasis"><em>Create the Master Yocto Controller</em></span>:
                        </p><pre class="literallayout">
     $ buildbot create-master <em class="replaceable"><code>yocto-controller</code></em>
                        </pre><p>
                        </p></li><li class="listitem"><p>
                        <span class="emphasis"><em>Change Your Working Directory to the Master Yocto Controller</em></span>:
                        </p><pre class="literallayout">
     $ cd <em class="replaceable"><code>yocto-controller</code></em>
                        </pre><p>
                        </p></li><li class="listitem"><p>
                        <span class="emphasis"><em>Create a Local Git Repository of the Yocto Project Autobuilder</em></span>:
                        </p><pre class="literallayout">
     $ git clone https://git.yoctoproject.org/git/yocto-autobuilder2 yoctoabb
                        </pre><p>
                        In the previous command, the local repository is
                        created in a <code class="filename">yoctoabb</code>
                        directory inside the directory of the Master
                        Yocto Controller directory.
                        </p></li><li class="listitem"><p>
                        <span class="emphasis"><em>Change Your Working Directory Back to the Master Yocto Controller</em></span>:
                        </p><pre class="literallayout">
     $ cd ..
                        </pre><p>
                        </p></li><li class="listitem"><p>
                        <span class="emphasis"><em>Create a Relative Symbolic Link to <code class="filename">master.cfg</code></em></span>:
                        </p><pre class="literallayout">
     $ ln -rs <em class="replaceable"><code>yocto-controller</code></em>/yoctoabb/master.cfg <em class="replaceable"><code>yocto-controller</code></em>/master.cfg
                        </pre><p>
                        The previous command sets up a relative symbolic
                        link to the <code class="filename">master.cfg</code> using
                        a link of the same name.
                        </p></li><li class="listitem"><p>
                        <span class="emphasis"><em>Update the Buildbot URL in <code class="filename">master.cfg</code></em></span>:
                        Use your <code class="filename">$EDITOR</code> to edit the
                        Buildbot URL in the <code class="filename">master.cfg</code>
                        file.
                        Find the following line and replace the URL with
                        the URL for your Buildbot:
                        </p><pre class="literallayout">
     c['buildbotURL'] = "https://autobuilder.yoctoproject.org/main/"
                        </pre><p>
                        </p></li><li class="listitem"><p>
                        <span class="emphasis"><em>Enable services in <code class="filename">services.py</code></em></span>:
                        Use your <code class="filename">$EDITOR</code> to edit the
                        <code class="filename">services.py</code> file.
                        Set appropriate configuration values to enable
                        desired services.
                        </p></li><li class="listitem"><p>
                        <span class="emphasis"><em>Enable Automatic Authorization (Autorisation) in <code class="filename">www.py</code></em></span>:
                        Use your <code class="filename">$EDITOR</code> to edit the
                        <code class="filename">www.py</code> file.
                        Configure autorisation if desired.
                        </p></li><li class="listitem"><p>
                        <span class="emphasis"><em>Modify Configuration Options in <code class="filename">config.py</code></em></span>:
                        Use your <code class="filename">$EDITOR</code> to edit the
                        <code class="filename">config.py</code> file.
                        Modify configuration options such as worker
                        configurations.
                        </p></li><li class="listitem"><p>
                        <span class="emphasis"><em>Start Buildbot</em></span>:
                        </p><pre class="literallayout">
     $ buildbot start <em class="replaceable"><code>yocto-controller</code></em>
                        </pre><p>
                        </p></li><li class="listitem"><p>
                        <span class="emphasis"><em>Create a Local Git Repository of the Yocto Autobuilder Helper Scripts:</em></span>:
                        </p><pre class="literallayout">
                        Move up a directory so that you are above the
                        <em class="replaceable"><code>yocto-controller</code></em>
                        location and clone the directory:
     $ cd ..
     $ git clone https://git.yoctoproject.org/git/yocto-autobuilder-helper
                        </pre><p>
                        </p></li></ol></div><p>
            </p></div><div class="section" title="1.8.2. Upstream Autobuilder Deployment on the Worker"><div class="titlepage"><div><div><h3 class="title" id="test-upstream-autobuilder-deployment-on-the-worker">1.8.2. Upstream Autobuilder Deployment on the Worker<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-upstream-autobuilder-deployment-on-the-worker">¶</a></span></h3></div></div></div><p>
                Follow these steps to deploy Yocto Autobuilder on an
                upstream worker:
                </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
                        <span class="emphasis"><em>Create the Worker</em></span>:
                        </p><pre class="literallayout">
     $ buildbot-worker create-worker <em class="replaceable"><code>yocto-worker</code></em> <em class="replaceable"><code>localhost</code></em> <em class="replaceable"><code>example-worker</code></em> <em class="replaceable"><code>pass</code></em>
                        </pre><p>
                        </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
                            You do not have to hard-code the third
                            parameter (i.e.
                            <em class="replaceable"><code>example-worker</code></em>).
                            For example, you can pass
                            <code class="filename">`hostname`</code> to use the
                            host's configured name.
                        </div><p>
                        </p></li><li class="listitem"><p>
                        <span class="emphasis"><em>Start the Worker</em></span>:
                        </p><pre class="literallayout">
     $ buildbot-worker start <em class="replaceable"><code>yocto-worker</code></em>
                        </pre><p>
                        </p></li></ol></div><p>
            </p></div><div class="section" title="1.8.3. Upstream Autobuilder Deployment No Upstream Users"><div class="titlepage"><div><div><h3 class="title" id="test-upstream-autobuilder-deployment-no-upstream-users">1.8.3. Upstream Autobuilder Deployment No Upstream Users<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-upstream-autobuilder-deployment-no-upstream-users">¶</a></span></h3></div></div></div><p>
                This case has yet to be defined.
                It requires a custom <code class="filename">config.json</code> file
                for <code class="filename">yocto-autobuilder-helper</code>.
            </p></div></div><div class="section" title="1.9. Setting Up Headless Sanity Tests"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-headless-sanity-tests">1.9. Setting Up Headless Sanity Tests<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-headless-sanity-tests">¶</a></span></h2></div></div></div><p>
            If you plan on using the Yocto Project Autobuilder to run
            headless sanity testing, you need to do the following:
            </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
                    Install
                    <a class="link" href="#test-tight-vnc">TightVNC</a>
                    client and server.
                    </p></li><li class="listitem"><p>
                    Create a bank of tap network devices (tap devs)
                    by running the
                    <code class="filename">runqemu-gen-tapdevs</code> script
                    found in the
                    <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#source-directory" target="_top">Source Directory</a>
                    at
                    <a class="ulink" href="https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts" target="_top">https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts</a>.
                    </p><p>You must disable interface control on these
                    new tap devices.
                    </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
                        Some services include NetworkManager,
                        connman, or wicd.
                    </div><p>
                    </p></li><li class="listitem"><p>
                    Add "xterm*vt100*geometry: 80x50+10+10" to
                    <code class="filename">.Xdefaults</code>
                    </p></li><li class="listitem"><p>
                    Set up and start the TightVNC session as the
                    Autobuilder user.
                    </p></li><li class="listitem"><p>
                    Manually connect to the VNC session at least once
                    prior to running a QEMU sanity test.
                    </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
                        Something is getting set during the initial
                        connection that has not been figured out yet.
                        Manually connecting seems to set up the session
                        correctly.
                    </div><p>
                    </p></li></ol></div><p>
        </p></div><div class="section" title="1.10. Adding Additional Build Workers"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-adding-additional-build-workers">1.10. Adding Additional Build Workers<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-adding-additional-build-workers">¶</a></span></h2></div></div></div><p>
            The production Yocto Autobuilder uses a cluster of build
            workers.
            The cluster shares the same
            <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-SSTATE_DIR" target="_top"><code class="filename">SSTATE_DIR</code></a>
            and
            <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-DL_DIR" target="_top"><code class="filename">DL_DIR</code></a>
            through an NFS4 mounted Network Attached Storage (NAS).
            The main nightly trigger pre-populates the
            <code class="filename">DL_DIR</code>, which allows the workers to not
            have to deal with a lot of downloading.
            In theory, you could also run your build workers with
            <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-NO_NETWORK" target="_top"><code class="filename">NO_NETWORK</code></a>
            to enforce a single point for populating
            <code class="filename">DL_DIR</code>.
        </p><p>
            Running multiple build workers is fairly simple, but does require
            some setup:
            </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
                    Ensure the settings in
                    <code class="filename">autobuilder.conf</code> are valid
                    for each worker.
                    Certain variables are set within this file that
                    work with the local configurations on each
                    worker.
                    </p></li><li class="listitem"><p>
                    Within
                    <code class="filename">yocto-controller/controller.cfg</code>,
                    add your worker to the
                    <code class="filename">c['workers']</code> list inside
                    the <code class="filename">BUILDWORKERS</code> section.
                    </p></li><li class="listitem"><p>
                    For each worker change the
                    <code class="filename">WORKER SETTINGS</code> section
                    of
                    <code class="filename">yocto-worker/buildbot.tac</code>
                    to match the settings in
                    <code class="filename">controller.cfg</code>.
                    </p></li><li class="listitem"><p>
                    Workers must reside in the same path as the
                    Build Controller, even if they are on
                    completely different machines.
                    </p></li></ol></div><p>
        </p></div><div class="section" title="1.11. Setting Up Build History"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-setting-up-build-history">1.11. Setting Up Build History<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-setting-up-build-history">¶</a></span></h2></div></div></div><p>
            Build History is used to track changes to packages and
            images.
            By default, the Autobuilder does not collect build history.
            The production Autobuilder does have this functionality
            enabled.
        </p><p>
            Setting up build history requires the following
            steps:
            </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
                    Create an empty Git repository.
                    Make a single commit to it and then create and
                    push branches for each of the nightly core
                    architectures (i.e.. mips, ppc, x86...).
                    </p></li><li class="listitem"><p>
                    Find a central location to create a clone for the
                    repository created in the previous step.
                    This works best if you have a setup similar to
                    the production Autobuilder (i.e. NAS with many
                    workers).
                    </p></li><li class="listitem"><p>
                    Run the following:
                    </p><pre class="literallayout">
     # This is an example of how to set up a local build history checkout. Paths
     # obviously are situationally dependent.
     $ mkdir /nas/buildhistory
     $ cd /nas/buildhistory
     $ git clone ssh://git@git.myproject.org/buildhistory
     $ git clone ssh://git@git.myproject.org/buildhistory nightly-arm
     $ git clone ssh://git@git.myproject.org/buildhistory nightly-x86
     $ git clone ssh://git@git.myproject.org/buildhistory nightly-x86-64
     $ git clone ssh://git@git.myproject.org/buildhistory nightly-ppc
     $ git clone ssh://git@git.myproject.org/buildhistory nightly-mips
     $ for x in `ls|grep nightly` do cd $x; git checkout $x; cd /nas/buildhistory; done
                    </pre><p>
                    </p></li><li class="listitem"><p>
                    Within the <code class="filename">autobuilder.conf</code>
                    of each worker, change the following:
                    </p><pre class="literallayout">
     BUILD_HISTORY_COLLECT = True
     BUILD_HISTORY_DIR = "/nas/buildhistory"
     BUILD_HISTORY_REPO = "ssh://git@git.myproject.org/buildhistory"
                    </pre><p>
                    </p></li></ol></div><p>
        </p></div><div class="section" title="1.12. Some More Notes"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-some-more-notes">1.12. Some More Notes<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-some-more-notes">¶</a></span></h2></div></div></div><p>
            </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
                    <span class="emphasis"><em>Yocto Autobuilder</em></span>:
                    The Git repository is at
                    <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/" target="_top">http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/</a>.
                    </p><p>Essentially an extension to the vanilla buildbot.
                    This extension mainly addresses configuration file handling
                    and Yocto-specific build steps.</p><p>For better maintainability, the Autobuilder (see
                    <code class="filename">Autobuilder.py</code> located at
                    <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/lib/python2.7/site-packages/autobuilder" target="_top">http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/lib/python2.7/site-packages/autobuilder</a>),
                    handles configuration from multiple files.</p><p>Additional build steps such as
                    <code class="filename">CheckOutLayers.py</code> or
                    <code class="filename">CreateBBLayersConf</code> are Yocto-specific
                    and simplify the worker's configuration.
                    </p></li><li class="listitem"><p><a id="test-tight-vnc"></a>
                    <span class="emphasis"><em>TightVNC</em></span>:
                    Virtual Network Computing (VNC) is a client/server software
                    package that allows remote network access to graphical
                    desktops.
                    With VNC, you can access your machine from everywhere
                    provided that your machine is connected to the Internet.
                    VNC is free (released under the GNU General Public License)
                    and it is available on most platforms.</p><p>TightVNC is an enhanced version of VNC, which
                    includes new features, improvements, optimizations, and
                    bug fixes over the original VNC version.
                    See the list of features at
                    <a class="ulink" href="http://www.tightvnc.com/intro.php" target="_top">http://www.tightvnc.com/intro.php</a>.
                    </p><p>You need TightVNC in order to run headless sanity
                    tests.
                    See the bullet on
                    <a class="link" href="#test-headless-sanity-tests" title="1.9. Setting Up Headless Sanity Tests">headless sanity tests</a>
                    for more information.
                    </p></li><li class="listitem"><p>
                    <span class="emphasis"><em>Files Used for Yocto-Autobuilder Configuration:</em></span>
                    </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
                            <span class="emphasis"><em><code class="filename">config/autobuilder.conf</code></em></span>:
                            Used to set Autobuilder-wide parameters, such as
                            where various build artifacts are published
                            (e.g. <code class="filename">DL_DIR</code> and
                            <code class="filename">SSTATE_DIR</code>).
                            Another example is if build artifacts should be
                            published, which is necessary for production
                            Autobuilders but not desktop builders.
                            </p></li><li class="listitem"><p>
                            <span class="emphasis"><em><code class="filename">buildset-config/yoctoAB.conf</code></em></span>:
                            The main Yocto Project Autobuilder configuration
                            file.
                            Documentation for this file and its associated
                            format is in the
                            <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/README-NEW-AUTOBUILDER" target="_top"><code class="filename">README-NEW-AUTOBUILDER</code></a>
                            file.
                            </p></li></ul></div><p>
                    </p></li></ul></div><p>
        </p></div><div class="section" title="1.13. Yocto Project Autobuilder Helper Scripts"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-yocto-project-helper">1.13. Yocto Project Autobuilder Helper Scripts<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-yocto-project-helper">¶</a></span></h2></div></div></div><div class="note" title="WRITER NOTE" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">WRITER NOTE</h3>
            Deferring this topic per Richard's suggestion.
            It is placed here temporarily.
        </div><p>
            The helper scripts work in conjunction with the Yocto Project
            Autobuilder.
            These scripts do the actual build configuration and execution
            for tests on a per release basis.
        </p><p>
            You can use <code class="filename">pre-commit-hook.sh</code> to verify
            the JSON file before committing it.
            Create a symbolic link as follows:
            </p><pre class="literallayout">
     $ ln -s ../../scripts/pre-commit-hook.sh .git/hooks/pre-commit
            </pre><p>
        </p><p>
            Most users will have to customize the helper script repository
            to meet their needs.
            The repository is located at
            <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper" target="_top">http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper</a>.
            The scripts themselves should be more generically reusable.
            The <code class="filename">config.json</code> is less reusable as it
            represents the Yocto Project Autobuilder test matrix.
        </p><p>
            Two customization options are possible: 1) variable substitution,
            and 2) overlaying configuration files.
            The standard <code class="filename">config.json</code> minimally attempts
            to allow substitution of the paths.
            The helper script repository includes a
            <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree/local-example.json" target="_top"><code class="filename">local-example.json</code></a>
            to show how you could override these from a separate configuration
            file.
            Pass the following into the environment of the autobuilder:
            </p><pre class="literallayout">
     ABHELPER_JSON="config.json local-example.json"
            </pre><p>
            As another example, you could also pass the following into the
            environment:
            </p><pre class="literallayout">
     ABHELPER_JSON="config.json /some/location/local.json"
            </pre><p>
        </p></div></div>

</div></body></html>